Solving Markov Decision Processes by Graphical Modelling: the DT-Planner Program

In this paper we will describe a software package , called DT-Planner, able to represent and solve nite-state Markov Decision Processes, by exploiting a novel graphical formalism, called Innuence View. An Innuence View is a directed acyclic graph that depicts the probabilistic relationships between the problems state variables in a generic time transition ; additional variables, called event variables , may be added, in order to describe the conditional independencies between state variables. By using the speciied conditional independence structure, an Innuence View may allow a parsimonious speciication of a Markov Decision Process. DT-Planner lets the user specify and manage models through a user-friendly graphical interface, and implements eecient for policy determination algorithms .